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August  16,  2004 

MEMORANDUM  FOR  ASSISTANT  SECRETARY  OF  THE  AIR  FORCE 

(FINANCIAL  MANAGEMENT  AND  COMPTROLLER) 

SUBJECT :  Report  on  Implementation  of  the  DoD  Management  Control  Program  for 
Air  Force  Acquisition  Category  11  and  m  Programs 
(Report  No.  D-2004- 108) 


We  are  providing  this  report  for  review  and  comment.  DoD  Directive  7650.3 
requires  that  all  recommendations  be  resolved  promptly.  Because  the  Air  Force  did  not 
comment  to  the  July  1,  2004,  draft  report,  we  request  that  the  Air  Force  provide 
comments  on  this  final  report  by  September  17,  2004. 

Management  comments  should  indicate  concurrence  or  nonconcurrence  with  the 
findings  and  each  recommendation.  Comments  must  describe  actions  taken  or  planned  in 
response  to  agreed-upon  recommendations  and  provide  the  completion  dates  of  the 
actions.  State  specific  reasons  for  any  nonconcurrence  and  propose  alternative  actions,  if 
appropriate.  If  possible,  please  send  management  comments  in  electronic  format  (Adobe 
Acrobat  file  only)  to  Audam@dodig.osd.mil.  Copies  of  the  management  comments  must 
contain  the  actual  signature  of  the  authorizing  official.  We  cannot  accept  the  /  Signed  / 
symbol  in  place  of  the  actual  signature.  If  you  arrange  to  send  classified  comments 
electronically,  they  must  be  sent  over  the  SECRET  Internet  Protocol  Router  Network 
(SPRNET). 

We  appreciate  the  courtesies  extended  to  the  staff.  Questions  should  be  directed 
to  Mr.  John  E.  Meling  at  (703)  604-9091  (DSN  664-9091)  or  Ms.  Celia  J.  Harrigan  at 
(703)  604-9092  (DSN  664-9092).  See  Appendix  C  for  the  report  distribution.  The  team 
members  are  listed  inside  the  back  cover. 


By  direction  of  the  Deputy  Inspector  General  for  Auditing: 


Mary  L.  Ugdne 
Assistant  hispector  General 
for  Acquisition  Management 
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Report  No,  D2004-108 
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August  16,  2004 


Implementation  of  the  DoD  Management  Control  Program  for 
Air  Force  Acquisition  Category  II  and  III  Programs 


Executive  Summary 


Who  Should  Read  This  Report  and  Why?  Civil  service  and  uniformed  officers  who 
are  involved  in  the  Air  Force  acquisition  decision  making  process  should  read  this  report 
because  it  provides  an  assessment  of  how  the  Air  Force  can  improve  its  management 
control  program  for  oversight  of  Air  Force  weapon  system  programs. 

Background.  This  audit  is  the  second  in  a  series  of  audits  to  evaluate  the  effectiveness 
of  the  Military  Departments’  management  control  programs  in  timely  identifying 
deviations  in  cost,  schedule,  and  performance  requirements  in  acquisition  program 
baselines  for  Acquisition  Category  II  and  III  programs  to  the  milestone  decision  authority 
and  in  identifying  whether  program  managers  are  reporting  their  status  on  preparing  and 
obtaining  approval  of  required  program  documentation  to  milestone  decision  authorities 
before  milestone  decision  reviews. 

The  first  report.  Inspector  General,  DoD  Report  No.  D-2004-47,  “Implementation  of  the 
DoD  Management  Control  Program  for  Army  Acquisition  Category  II  and  III  Programs,” 
January  23,  2004,  assessed  the  Army  Management  Control  Program  for  oversight  of 
Acquisition  Category  II  and  III  programs.  The  third  report.  Report  No.  D-2004-I09, 
“Implementation  of  the  DoD  Management  Control  Program  for  Navy  Acquisition 
Category  II  and  III  Programs,”  August  17,  2004,  assessed  the  Navy  Management  Control 
Program  for  oversight  of  Acquisition  Category  II  and  III  programs. 

Results,  The  Air  Force  did  not  effectively  integrate  the  requirements  of  the  DoD 
Management  Control  Program  into  its  management  assessment  and  reporting  process  for 
eight  Acquisition  Category  II  and  III  programs  that  had  an  estimated  life-cycle  cost  of 
$1.9  billion.  As  a  result,  the  program  managers  did  not  have  documented  and  updated 
information  needed  to  more  effectively  manage  their  programs.  The  program  managers 
further  contributed  to  this  condition  by  not  reporting  program  deviations  and  not 
requesting  revisions  to  acquisition  program  baselines  when  cost,  schedule,  and 
performance  breaches  occurred.  Program  managers  did  not  provide  milestone  decision 
authorities  with  timely  and  documented  information  so  they  could  assist  program 
managers  who  were  experiencing  cost  overruns,  schedule  delays,  and  performance 
problems.  Further,  milestone  decision  authorities  made  important  program  decisions  at 
milestone  decision  points  with  incomplete  information  on  the  readiness  of  the  systems  for 
the  next  phase  of  the  acquisition  process.  To  ensure  that  Air  Force  milestone  decision 


authorities  make  fully  informed  investment  deeisions,  the  Air  Foree  needs  to  modify  its 
existing  reporting  requirements  for  program  managers  of  Aequisition  Category  II  and  III 
programs  to  report,  at  least  quarterly,  their  progress  toward  satisfying  eost,  sehedule,  and 
performanee  requirements  in  aequisition  program  baselines  and  obtaining,  preparing  and 
updating,  and  issuing  approved  program  doeumentation  before  and  at  planned  milestone 
deeision  reviews.  Implementation  of  this  reporting  requirement  will  enable  Air  Foree 
milestone  deeision  authorities  to  ensure  that  program  managers  are  on  schedule  toward 
satisfying  program  requirements,  submitting  program  deviation  reports  when  required, 
and  reporting  deviations  from  acquisition  program  baselines  as  management  control 
weaknesses.  (See  the  Finding  section  for  details.) 

Management  Comments.  The  Air  Force  did  not  respond  to  the  July  1,  2004,  draft 
report.  Therefore,  we  request  that  the  Air  Force  Acquisition  Executive  provide 
comments  on  this  final  report  by  September  17,  2004. 
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Background 


This  audit  is  the  second  in  a  series  of  audits  that  will  evaluate  the  effectiveness  of 
the  Military  Departments’  management  control  programs  in  timely  identifying 
deviations  in  cost,  schedule  and  performance  requirements  in  acquisition  program 
baselines  (APB)  for  Acquisition  Category  II  and  III  programs^  to  the  milestone 
decision  authority  (MDA)  and  in  identifying  whether  program  managers  are 
preparing  and  obtaining  required  program  documentation  before  milestone 
decision  reviews.  DoD  Instruction  5000.2,  “Operation  of  the  Defense  Acquisition 
System,”  May  12,  2003,  defines  Acquisition  Category  II  programs  as  major 
systems  with  an  estimated  total  expenditure  for  research,  development,  test  and 
evaluation  of  less  than  $365  million  but  more  than  $140  million,  or  for 
procurement,  of  less  than  $2.19  billion  but  more  than  $660  million  in  FY  2000 
constant  dollars.  DoD  Instruction  5000.2  defines  an  Acquisition  Category  III 
program  as  any  program  that  does  not  meet  the  minimum  requirements  of  an 
Acquisition  Category  II  program.  The  first  report.  Inspector  General,  DoD 
Report  No.  D-2004-47,  “Implementation  of  the  DoD  Management  Control 
Program  for  Army  Acquisition  Category  II  and  III  Programs,”  January  23,  2004, 
assessed  the  Army  Management  Control  Program  for  oversight  of  Acquisition 
Category  II  and  III  programs.  The  third  report.  Inspector  General,  DoD  Report 
No.  D-2004-109,  “Implementation  of  the  DoD  Management  Control  Program  for 
Navy  Acquisition  Category  II  and  III  Programs,”  August  17,  2004,  assessed  the 
Navy  Management  Control  Program  for  oversight  of  Acquisition  Category  II  and 
III  programs. 

Assistant  Secretary  of  the  Air  Force  Emphasizes  Management  Control 
Process.  The  Assistant  Secretary  of  the  Air  Force  (Financial  Management  and 
Comptroller)  stressed  the  importance  of  management  controls  in  a  February  25, 
2001,  memorandum  on  stewardship  and  the  Air  Force  management  control 
process.  The  Assistant  Secretary  emphasized  that  strong  management  controls 
should  help  to  ensure  that  resources  are  used  efficiently  and  effectively  to  achieve 
the  intended  program  results.  The  Assistant  Secretary  stated  that  management 
controls  are  not  an  isolated  tool  but  development  strategies  for  reengineering  Air 
Force  programs  in  the  years  ahead.  The  Assistant  Secretary  went  on  to  say  that 
management  controls  are  an  integral  part  of  the  entire  planning,  budgeting, 
management,  accounting  and  reporting  cycle. 

Management  Control  Policy.  Office  of  Management  and  Budget  (0MB) 
Circular  A- 123,  “Management  Accountability  and  Control,”  June  21,  1995, 


*An  acquisition  category  determines  an  acquisition  program’s  level  of  review,  decision  authority,  and 
applicable  procedures.  The  acquisition  categories  consist  of  I,  major  Defense  acquisition  programs;  lA, 
major  automated  information  systems;  II,  major  systems;  and  III,  programs  not  meeting  the  criteria  for 
acquisition  categories  I,  lA,  or  II. 
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defines  management  controls  as  the  organization,  policies,  and  procedures  that 
agencies  use  to  ensure  that  programs  achieve  their  intended  results;  resources  are 
used  consistent  with  an  organization’s  mission;  programs  and  resources  are 
protected  from  fraud,  waste,  and  mismanagement;  laws  and  regulations  are 
followed;  and  reliable  data  are  obtained,  maintained,  reported,  and  used  for 
program  decision  making.  Further,  the  DoD  and  the  Air  Force  have  issued 
guidance  to  implement  the  requirements  of  0MB  Circular  A- 123. 

Federal  Policy.  0MB  Circular  A-123  requires  DoD  to  provide  an  overall 
annual  statement  to  the  President  and  Congress  explaining  the  state  of  DoD 
management  controls,  any  control  weaknesses,  and  noncompliant  accounting 
systems.  The  Secretary  of  Defense  is  required  to  state  whether  DoD  management 
controls  provide  reasonable  assurance  that  resources  are  being  protected  from 
fraud,  waste,  and  mismanagement.  Circular  A-123  provides  guidance  for 
establishing,  evaluating,  improving,  and  reporting  on  management  controls  in 
programs  and  administrative  organizations.  Further,  it  directs  that  executive 
organizations  establish  cost-effective  management  control  systems  to  provide 
reasonable  assurance  that  resources  are  protected  against  fraud,  waste,  and 
mismanagement  and  that  program  activities  be  effectively  and  efficiently 
managed  to  achieve  the  goals  of  the  organization.  Circular  A-123  also  states  that 
Federal  managers  must  carefully  consider  the  appropriate  balance  of  controls  in 
their  programs  and  operations.  Appropriate  management  controls  should  be 
established  and  integrated  into  each  system  to  direct  and  guide  operations. 
However,  Circular  A-123  notes  that  a  separate  management  control  process  need 
not  be  instituted,  particularly  if  its  sole  purpose  is  to  satisfy  management  control 
reporting  requirements. 

DoD  Policy.  To  satisfy  the  0MB  requirement,  DoD  Directive  5010.38, 
“Management  Control  (MC)  Program,”  August  26,  1996,  requires  that  the 
Military  Departments,  the  Defense  agencies,  and  the  major  joint  commands 
submit  annual  statements  on  their  management  controls  to  the  Secretary  of 
Defense.  There  are  no  exemptions  for  acquisition  programs.  DoD 
Directive  5010.38  also  provides  policy,  prescribes  procedures,  and  assigns 
responsibilities  for  management  control  systems.  DoD  Directive  5000.1,  “The 
Defense  Acquisition  System,”  May  12,  2003,  requires  program  managers  to 
establish  program  goals  for  cost,  schedule,  and  performance  parameters  that 
describe  the  program  over  its  life  cycle.  Directive  5000.1  further  states  that 
approved  program  baseline  parameters  will  serve  as  control  objectives,  and  that 
the  program  manager  is  required  to  identify  deviations  from  approved  program 
baseline  parameters  and  exit  criteria.  Further,  the,  “DoD  Interim  Defense 
Acquisition  Guidebook,”  October  30,  2002,  states  that  the  program  manager 
should  identify  deviations  from  the  approved  parameters  and  exit  criteria  as 
material  weaknesses. 
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Air  Force  Policy.  Air  Force  Policy  65-201,  “Management  Control,” 

May  1,  1997,  implements  publie  law  and  0MB  and  DoD  guidance  for  a 
management  control  process,  and  requires  Air  Force  commanders  and  managers 
to  maintain  an  effeetive  eontrol  proeess  that  identifies,  reports,  and  corrects 
material  weaknesses. 

Scope  of  Air  Force  Programs  Reviewed.  To  perform  the  audit,  we  coordinated 
with  seven  Air  Force  Program  Executive  Offices  to  identify  Aequisition 
Category  II  and  III  programs  for  our  review  that  were  beyond  the  coneept  and 
teehnology  development  phase,  but  that  had  not  received  approval  for  full-rate 
production.  The  Aequisition  Category  II  systems  identified  included  the 
Integrated  Broadcast  Service,  the  Family  of  Advaneed  Beyond-Line-of-Sight 
Terminals,  and  the  Miniature  Air  Launch  Decoy.  The  Aequisition  Category  III 
systems  identified  included  the  Command  and  Control  System-Consolidated,  Air 
Foree  Terminal  Instrument  Procedures-Replaeement,  Common  Low  Observable 
Verifieation  System,  Universal  Munitions  Trailer,  and  L-15E  Advanee  Display 
Core  Proeessor. 


Objectives 

The  overall  audit  objective  was  to  evaluate  the  effeetiveness  of  the  management 
eontrol  program  that  the  Air  Lorce  Aequisition  Executive,  or  his  designees,  used 
for  Air  Eoree  Aequisition  Category  II  and  III  weapon  systems.  We  also  assessed 
the  proeess  for  reeognizing  and  correcting  material  management  control 
weaknesses.  See  Appendix  A  for  a  discussion  of  the  scope  and  methodology  and 
the  review  of  the  management  control  program.  See  Appendix  A  for  prior 
coverage  related  to  the  objectives.  Appendix  B  defines  management  control 
objectives  and  teehniques.  Appendix  C  summarizes  the  status  of  certain  program 
doeumentation  prepared  for  the  eight  programs  reviewed. 
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Air  Force  Management  Controls  for 
Acquisition  Category  II  and  III  Programs 

The  Air  Force  did  not  effectively  integrate  the  requirements  of  the  DoD 
Management  Control  Program  into  its  management  assessment  and 
reporting  process  for  eight  Acquisition  Category  II  and  III  programs  that 
had  an  estimated  life-cycle  cost  of  $  1 .9  billion.  This  condition  occurred 
because  the  existing  reporting  mechanism  in  the  Air  Force  did  not  require 
program  managers  to  inform  MDAs,  at  least  quarterly,  of  their  progress 
toward: 


•  satisfying  cost,  schedule,  and  performance  requirements  in  APBs; 
and 

•  obtaining,  preparing,  updating,  and  issuing  approved  program 
documentation  before  and  at  planned  milestone  decision  reviews. 

As  a  result,  the  program  managers  did  not  have  documented  and  updated 
information  needed  to  more  effectively  manage  their  programs.  The 
program  managers  further  contributed  to  this  condition  by  not  reporting 
program  deviations  and  not  requesting  revisions  to  APBs  when  cost, 
schedule,  and  performance  breaches  occurred.  As  a  result,  program 
managers  did  not  provide  MDAs  with  timely  and  documented  information 
so  they  could  assist  program  managers  who  were  experiencing  cost 
overruns,  schedule  delays,  and  performance  problems.  Further,  MDAs 
made  important  program  decisions  at  milestone  decision  points  with 
incomplete  information  on  the  readiness  of  the  systems  for  the  next  phase 
of  the  acquisition  process. 


Air  Force  Management  Control  Program 


On  July  23,  2003,  the  Office  of  the  Secretary  of  the  Air  Force  restructured  the  Air 
Force  Program  Executive  Officers  reporting  process.  The  Air  Force  management 
structure  intended  the  process  to  reduce  workload  and  promote  ownership  and 
accountability  for  effective  management  controls  by: 

•  developing  and  implementing  appropriate,  cost  effective  management 
controls, 

•  assessing  the  adequacy  of  management  controls  in  all  programs  and 
operations. 
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•  identifying  needed  improvements, 

•  taking  eorresponding  eorrective  action,  and 

•  reporting  annually  on  management  controls. 

Air  Force  Management  Control  Program  Requirements,  The  Assistant 
Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller)  developed 
the  Air  Force’s  management  control  program  based  on  the  General  Accounting 
Office’s  five  standards  for  internal  control:  control  environment,  risk  assessment, 
control  activities,  communication  and  information,  and  monitoring.  All  Air  Force 
commands  are  required  to  establish  a  management  control  program  and 
continuously  monitor  and  improve  the  effectiveness  and  accountability  of 
management  controls  associated  with  their  programs.  Each  command  is  to  update 
its  written  plan  annually,  and  include  information  that  indicates  the  number  of 
scheduled  and  accomplished  management  control  evaluations,  the  inventory  of 
assessable  units,  progress  toward  accomplishment  of  annual  program 
requirements,  the  method  of  monitoring  and  evaluation,  and  the  completion  date 
of  the  evaluation.  Emphasis  is  placed  on  maximizing  the  use  of  existing 
management  evaluation  data  and  minimizing  the  creation  of  processes  or 
procedures  solely  to  execute  the  management  control  program.  It  is  the 
responsibility  of  the  Management  Control  Program  Coordinator  to  prepare  the 
organization’s  management  control  plan  and  annual  management  control 
statement. 

Management  Controls  for  Acquisition  Programs,  Program  documentation 
was  a  key  component  in  the  management  control  plans  provided  by  the  eight 
Acquisition  Category  II  and  III  program  offices.  Milestone  documentation  is 
relied  on  as  one  of  the  key  management  controls  for  acquisition  programs,  and 
the  milestone  decision  review  process  for  evaluating  controls  was  in  place.  The 
Air  Eorce’s  process  to  evaluate  management  controls  for  acquisition  programs 
did  not,  however,  keep  MDAs  informed  of  the  program  managers’  progress 
toward  satisfying  program  requirements  before  milestone  decision  reviews. 
Accordingly,  the  Air  Eorce’s  Management  Control  Program  for  acquisition 
programs  did  not  ensure  that  program  managers  took  the  necessary  actions  to 
satisfy  the  program’s  cost,  schedule,  and  performance  requirements  in  the  APBs 
before  milestone  reviews.  The  Air  Eorce’s  Management  Control  Program  also 
did  not  ensure  that  program  managers  reported  deviations  from  APB  cost, 
schedule,  and  performance  parameters,  or  that  those  breaches  were  reported  as 
management  control  weaknesses  to  the  Assistant  Secretary  of  the  Air  Eorce 
(Einancial  Management  and  Comptroller). 
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Air  Force  Reporting  Process 


The  existing  reporting  meehanism  in  the  Air  Foree  did  not  require  program 
managers  for  Aequisition  Category  II  and  III  programs  to  inform  the  MDA,  at 
least  quarterly,  of  their  progress  towards  satisfying  eost,  sehedule,  and 
performanee  requirements  in  approved  APBs  and  of  exit  eriteria  requirements  for 
the  next  phase  of  the  aequisition  proeess.  The  Air  Force  has  more  than  1,400 
Acquisition  Category  II  and  III  programs.  Without  a  reporting  mechanism  in 
place,  the  Air  Force  MDAs  cannot  provide  appropriate  oversight  for  those 
programs.  Although  Acquisition  Category  II  and  III  programs  are  valued  less 
than  Acquisition  Category  I  programs,  the  sheer  number  of  Acquisition  Category 
II  and  III  programs  contributes  to  a  large  aggregate  total.  Given  funding 
constraints,  the  Air  Force  needs  to  manage  wisely  its  overall  investment  dollars 
and  cannot  afford  to  overlook  the  oversight  of  the  Acquisition  Category  II  and  III 
programs. 

The  reporting  process  used  by  the  Program  Executive  Officers  did  not  require  the 
Air  Force  program  managers  to  report  on  their  efforts  to  obtain,  prepare,  and 
update  approved  program  documentation  before  planned  milestone  decision 
reviews.  The  program  documentation  is  to  be  prepared  and  approved  before 
milestone  decision  points,  kept  up-to-date  between  milestone  decision  reviews, 
and  provide  the  MDA  with  the  management  control  information  required  in  DoD 
Directive  5000.1  to  timely  identify  program  deviations  from  approved  APB 
parameters  and  exit  criteria.  As  implemented,  the  Air  Force’s  management 
assessment  and  reporting  processes  did  not  provide  MDAs  with  that  management 
control  information.  To  be  effective,  the  Air  Force’s  management  control 
program  should  integrate  those  management  control  requirements  in  its 
management  assessment  and  reporting  processes. 

Reporting  on  Cost,  Schedule,  and  Performance  Requirements.  Program 
managers  for  major  DoD  acquisition  programs  are  required  to  report  quarterly  on 
their  progress  towards  satisfying  cost,  schedule,  and  performance  requirements, 
including  approved  program  baseline  parameters  and  exit  criteria  requirements,  in 
the  Defense  Acquisition  Executive  Summary.  On  April  2,  1999,  the  Principal 
Deputy  Assistant  Secretary  (Acquisition  and  Management)  began  requiring  all 
Acquisition  Category  II  and  III  programs  encountering  programmatic  issues  to 
submit  monthly  status  reports,  and  required  the  remaining  programs  to  submit 
quarterly  status  reports.  On  December  11,  2002,  the  Deputy  Assistant  Secretary 
(Management  Policy  and  Program  Integration)  required  all  program  offices  to 
implement  the  system  metric  and  reporting  tool.  A  key  feature  of  the  system 
metric  and  reporting  tool  is  to  provide  the  MDA  with  information  included  in  a 
monthly  acquisition  report.  Although  the  memorandum  requires  program 
managers  to  submit  monthly  and  quarterly  status  reports,  it  does  not  require 
program  managers  to  inform  MDAs,  at  least  quarterly,  of  their  progress  toward 
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satisfying  cost,  schedule,  and  performanee  requirements  in  the  aequisition 
program  baseline  and  exit  eriteria  requirements. 

Reporting  for  Acquisition  Category  II  Programs  Reviewed.  Program 
managers  for  the  three  Aequisition  Category  II  programs  reviewed  prepared 
monthly  aequisition  reports,  two  of  the  three  program  managers  prepared  program 
management  reviews,  and  one  of  the  program  managers  prepared  semi-annual 
portfolio  reviews  for  the  Program  Exeeutive  Offiee. 

Reporting  for  Acquisition  Category  III  Programs  Reviewed.  Program 
managers  for  the  five  Aequisition  Category  III  programs  reviewed  prepared 
monthly  aequisition  reports,  four  of  the  program  managers  prepared  for  program 
management  reviews,  and  three  of  the  program  managers  prepared  for  semi¬ 
annual  portfolio  reviews  for  the  Program  Exeeutive  Offiee. 

Adequacy  of  Reported  Information.  The  monthly  aequisition  reports, 
program  management  reviews,  and  Program  Executive  Offiee  semi-annual 
reviews  that  the  program  offiees  prepared  provided  the  MDA  with  up-to-date 
APB  eost,  sehedule,  and  performanee  data,  but  did  not,  in  most  oases,  provide  the 
APB  threshold  values  for  the  MDA  to  oompare  the  data  with.  Without  listing  the 
APB  thresholds  with  the  existing  program  status,  the  MDA  may  not  realize  that 
the  program  is  over  budget  or  behind  sehedule.  Additionally,  the  program  offiee 
reports  did  not  provide  the  MDA  with  information  on  whether  the  program  would 
aohieve  the  exit  eriteria  established  at  the  last  milestone  deoision  review.  The 
inolusion  of  the  APB  threshold  values  and  exit  eriteria  for  the  next  phase  of  the 
aequisition  proeess  in  the  program  managers’  reports  would  inform  the  MDAs  of 
deviations  from  approved  APB  parameters  and  exit  eriteria. 

Reporting  on  the  Preparation  and  Completion  of  Required  Program 
Documentation.  DoD  and  Air  Eoree  procedures  do  not  require  program 
managers  to  advise  MDAs  on  their  progress  towards  completing  and  obtaining 
key  program  documentation,  such  as  the  operational  requirements  document 
(ORD),  the  APB,  the  test  and  evaluation  master  plan  (TEMP),  and  the  command, 
control,  communications,  computers,  and  intelligence  (C4I)  support  plan  before 
program  initiation,  which  are  usually  required  when  a  program  enters  the  system 
development  and  demonstration  acquisition  phase.  At  program  initiation  and 
before  subsequent  program  milestone  reviews,  however,  DoD  and  Air  Eoree 
policy  requires  program  managers  to  provide  those  key  program  documents  for 
the  MDA  to  review. 

Eor  all  eight  programs  reviewed,  program  managers  held  milestone  reviews  with 
MDAs  for  entry  into  the  system  development  and  demonstration  phase  of  the 
acquisition  process  before  preparing,  updating,  or  obtaining  all  required 
documentation.  Eor  example,  the  Air  Eoree  Program  Executive  Office  for 
Aircraft  Systems  approved  the  E-15E  Advanced  Display  Core  Processor  to  enter 
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the  system  development  and  demonstration  phase  of  the  aequisition  process  in 
June  1999,  even  though  the  program  manager  did  not  have  an  approved  APB, 

C4I  support  plan.  Programmatic  Environmental,  Safety,  and  Health  Evaluation 
(PESHE)  or  TEMP.  The  ORD  identified  interoperability  as  a  key  performance 
parameter.  Because  the  program  manager  did  not  have  an  approved  C4I  support 
plan,  contract  requirements  defining  all  the  key  system  interfaces  with  other 
systems  may  not  have  been  identified  and  provided  to  the  system  contractor. 

Program  documentation  that  was  not  available  to  MDAs  at  milestone  reviews 
varied  significantly  among  the  programs  reviewed.  Eor  instance,  program 
managers  for  two  programs  did  not  have  an  approved  ORD,  program  managers 
for  seven  programs  did  not  have  an  approved  TEMP,  and  program  managers  for 
six  programs  did  not  have  an  approved  C4I  support  plan  available  at  their 
previous  milestone  review.  Without  an  approved  ORD  that  contains  validated 
system  performance  requirements,  the  program  manager  and  contractor  cannot  be 
sure  that  the  system  design  will  meet  warfighter-required  capabilities,  and  the  test 
community  does  not  have  validated  performance  parameters  for  developmental 
and  operational  testing.  An  approved  TEMP  is  an  essential  tool  for  the  program 
manager  and  operational  testers  to  identify  resources  needed  to  perform  necessary 
tests  and  to  test  all  key  performance  parameters.  The  C4I  support  plan  is  critical 
not  only  for  system  design,  but  for  the  test  organizations  to  identify  system 
integration  problems  that  need  to  be  corrected  during  testing.  Although 
acquisition  regulations  encourage  management  to  streamline  and  tailor  acquisition 
documentation,  the  ORD,  the  TEMP,  and  the  C4I  support  plan  must  be  prepared 
and  completed  to  provide  the  MDA  with  information  needed  to  assess  how  well  a 
program  is  progressing  toward  the  next  milestone  review  in  the  acquisition 
process. 

Updating  Program  Documentation  Between  Milestone  Decision  Reviews. 

Additionally,  program  managers  did  not  provide  MDAs  with  updated  program 
documentation  between  milestone  decision  reviews  when  significant  changes 
affected  their  programs  because  they  believed  that  updating  program 
documentation  added  no  value  to  the  process  when  MDAs  were  made  aware  of 
the  changes  through  other  means.  Eor  example,  the  Program  Manager  for  the 
Eamily  of  Advanced  Beyond-Eine-of-Sight  Terminals  experienced  significant 
cost  and  schedule  deviations  because  of  funding  cuts.  In  this  case,  the  program 
manager  did  not  submit  a  program  deviation  report  to  document  the  effects  on  the 
program,  partly  because  no  APB  had  been  prepared  to  measure  thresholds  against. 
Instead,  the  program  manager  communicated  information  on  cost  and  schedule 
breaches  through  quarterly  program  management  reviews,  monthly  activity 
reports,  and  weekly  activity  reports  to  the  MDA.  It  is  important  for  program 
managers  to  document  significant  changes  to  program  direction  in  cost,  schedule, 
and  performance  requirements  so  that  all  parties  affected  by  the  changes, 
including  the  intended  user  of  the  system  and  the  test  community,  can  plan  and 
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react  accordingly.  Updating  formal  program  documentation  is  essential  to  ensure 
that  the  program  manager  evaluates  the  effect  of  the  changes  on  all  aspects  of  the 
program. 

Appendix  C  describes  the  completeness  of  program  documentation  available  at 
milestone  decision  reviews  for  the  eight  programs  reviewed.  Without  having  all 
required  program  documentation,  the  program  manager  and  the  MDA  may  not  be 
aware  of  potential  or  actual  cost,  schedule,  and  performance  problems  affecting 
the  program  at  the  time  of  the  milestone  decision  review.  For  example,  the 
Family  of  Advanced  Beyond-Line-of-Sight  Terminals  experienced  schedule 
delays  and  cost  overruns  after  the  program  entered  the  system  development  and 
demonstration  phase  because  the  test  community  encountered  more  testing 
requirements  than  had  been  planned.  Because  the  TEMP  was  not  completed  and 
fully  reviewed  within  the  test  community,  the  program  manager’s  estimates  on 
the  time  and  resources  needed  for  testing  were  not  realistic. 

If  program  managers  of  the  eight  programs  reviewed  had  formally  reported  delays 
to  the  MDAs  in  the  completion  or  approval  process  for  key  program  documents, 
such  as  the  ORD  and  the  TEMP,  the  MDAs  could  have  initiated  actions  to 
prepare,  update,  and  approve  the  documentation  before  the  milestone  decision 
reviews,  or  to  reschedule  the  milestone  decision  reviews  pending  the  completion 
of  required  program  documentation.  Because  program  managers  were  not 
periodically  reporting  on  their  status  toward  completing  or  obtaining  required 
program  documentation,  MDAs  did  not  have  information  needed  to  take 
appropriate  actions  before  the  milestone  decision  reviews  for  the  eight  programs 
reviewed.  MDAs  also  were  not  requesting  information  concerning  the  status  of 
program  documentation  completion  from  program  managers  before  program 
milestone  reviews. 

Program  Deviation  Reporting  Requirements.  In  addition  to  not  routinely 
reporting  on  program  status  towards  satisfying  program  cost,  schedule,  and 
performance  parameters  in  APBs,  program  managers  for  five  of  the  eight 
acquisition  programs  did  not  have  APBs  to  compare  threshold  values  with  at  their 
last  milestone  decision  review;  therefore,  they  did  not  prepare  program  deviation 
reports  to  show  threshold  value  breaches.  Additionally,  two  program  managers 
with  APBs  did  not  prepare  program  deviation  reports  for  cost  breaches. 

Reporting  program  deviations  is  an  essential  communication  between  the  program 
managers  and  MDAs  because  it  serves  as  an  early  warning  of  impending  program 
complications  and  allows  the  MDA  to  help  with  a  program  solution.  If  program 
managers  do  not  submit  APBs,  the  program  deviations  can  go  unnoticed  by  the 
MDA  while  the  program  manager  attempts  to  bring  the  program  back  into 
control. 
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Although  program  managers  were  not  reporting  breaehes  of  APB  parameters  as 
management  control  weaknesses  as  required  in  DoD  Directive  5000.1  and  the 
DoD  Interim  Guidebook,  they  did  request  revisions  of  APB  parameters  when 
approaching  a  breach  of  a  parameter.  Therefore,  program  managers  did  not  report 
breaches  of  an  APB  parameter  as  a  management  control  weakness  to  the  Assistant 
Secretary  of  the  Air  Force  (Acquisition)  as  required.  As  a  result,  the  Assistant 
Secretary  of  the  Air  Force  (Acquisition)  did  not  consider  those  APB  parameter 
breaches  for  inclusion  as  a  material  management  control  weakness  in  the  FY  2003 
statement  of  assurance. 


Effect  of  Management  Control  Techniques  on  Program 
Documentation 

Program  documentation  is  the  primary  means  for  providing  the  MDA,  as  well  as 
other  key  managers,  with  information  needed  for  decision  making.  Without 
accurate  and  updated  program  documents,  such  as  the  APB,  the  ORD,  the  TEMP, 
and  the  C4I  support  plan,  along  with  the  program  manager’s  periodic  reporting  of 
progress  made  toward  satisfying  those  requirements,  the  program  manager  cannot 
provide  assurance  to  the  MDA  that  cost,  schedule,  and  performance  thresholds  in 
the  APB  are  being  achieved  and  that  the  program  is  affordable.  Without  this 
information,  program  managers  and  MDAs  did  not  have  documented  information 
needed  for  program  management  and  oversight,  and  MDAs  did  not  have 
information  needed  to  assist  program  managers  who  were  experiencing  program 
cost  overruns,  schedule  delays,  and  performance  problems.  Further,  MDAs  did 
not  have  the  information  they  needed  at  milestone  decision  points  to  make 
informed  program  decisions  on  the  readiness  of  the  systems  to  continue  into  the 
next  phase  of  the  acquisition  process. 


Conclusion 

The  management  control  program  for  Air  Force  Acquisition  Category  II  and  III 
programs  varied  depending  upon  the  commitment  of  program  management.  As  a 
result  of  the  Air  Force  program  offices  maximizing  the  use  of  existing 
management  evaluation  data  to  accomplish  their  management  control 
assessments,  the  assessments  did  not,  in  most  cases,  provide  timely  management 
control  information  to  MDAs  on  the  program  managers’  progress  toward 
satisfying  cost,  schedule,  and  performance  parameters  in  APBs,  which  are  the 
control  objectives  for  acquisition  programs  established  in  DoD  Directive  5000.1. 

Techniques  to  achieve  control  objectives  also  exist  in  the  preparation  and 
presentation  of  key  documents  such  as  the  APB  agreement,  earned  value 
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management  data,  and  operational  test  reports.  Aceordingly,  modifying  the 
existing  Air  Foree  reporting  requirements  for  Aequisition  Category  II  and  III 
programs  to  inelude  APB  threshold  values,  exit  eriteria,  and  program  performanee 
for  those  requirements  would  provide  Air  Force  MDAs  with  management  control 
information  to  implement  the  requirements  in  DoD  Directive  5000.1  and  would 
satisfy  the  management  control  reporting  requirements  in  DoD  Directive  5010.38. 

Because  of  the  conditions  identified  during  the  audit,  the  Air  Force  Acquisition 
Executive  should  implement  the  following  recommendations  to  improve  the  Air 
Force  existing  management  control  reporting  requirements  for  Acquisition 
Category  II  and  III  programs. 


Recommendations 

1 .  We  recommend  that  the  Air  Force  Acquisition  Executive  modify  its  existing 
reporting  requirements  for  program  managers  of  Acquisition  Category  II  and  III 
programs  to  report  to  the  milestone  decision  authority,  at  least  quarterly,  on  their 
progress  towards  satisfying  cost,  schedule,  and  performance  parameters  in 
approved  acquisition  program  baselines,  satisfying  exit  criteria  for  the  next  phase 
in  the  acquisition  process,  and  obtaining  and  preparing  key  program 
documentation,  such  as  the  operational  requirements  document;  the  test  and 
evaluation  master  plan;  and  the  command,  control,  computers,  communications, 
and  intelligence  support  plan  between  and  for  milestone  decision  reviews. 

2.  We  recommend  that  Air  Eorce  Program  Executive  Officers: 

a.  Use  information  in  the  program  managers’  progress  reports  discussed 
in  Recommendation  1 .  to  require  that  program  managers  prepare  program 
deviation  reports  when  breaches  of  acquisition  program  baseline  parameters 
occur. 


b.  Report  breaches  of  acquisition  program  baseline  parameters  and 
unattained  program  exit  criteria  to  the  Assistant  Secretary  of  the  Air  Eorce 
(Acquisition)  for  consideration  as  material  management  control  weaknesses  for 
inclusion  in  the  Air  Eorce  annual  statements  of  assurance  as  required  in  DoD 
Directive  5000.1,  “The  Defense  Acquisition  System,”  May  12,  2003,  and  DoD 
Directive  5010.38,  “Management  Control  (MC)  Program,”  August  26,  1996. 
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Appendix  A.  Scope  and  Methodology 


To  perform  the  audit,  we  coordinated  with  the  Air  Force  Program  Executive 
Offices  to  select  Acquisition  Category  II  and  III  programs  for  our  review  that 
were  beyond  the  concept  and  technology  development  phase,  but  had  not  entered 
the  production  and  deployment  phase  of  the  acquisition  process.  Accordingly,  we 
reviewed  the  following  eight  Air  Force  acquisition  programs  that  the  Air  Force 
Program  Executive  Offices  identified: 

PEO  Aircraft  Systems 

•  Common  Eow  Observable  Verification  Systems* 

•  F-15E  Advanced  Display  Core  Processor  Program* 

PEO  Command  and  Control 

•  Air  Force  Terminal  Instrument  Procedures-Replacement* 

•  Integrated  Broadcast  Service 
PEO  Space 

•  Command  and  Control  System-Consolidated* 

2 

•  Family  of  Advanced  Beyond-Line-of-Sight  Terminals 
PEO  Weapons 

2 

•  Miniature  Air  Eaunched  Decoy 

•  Universal  Munitions  Trailer* 

In  reviewing  the  management  control  process  used  by  the  Air  Force  for 
Acquisition  Category  II  and  III  programs,  we  examined  FY  2002  and  FY  2003 
management  control  documentation.  That  documentation  included  management 
control-related  tasking  from  the  Assistant  Secretary  of  the  Air  Force  (Financial 
Management  and  Comptroller),  the  Assistant  Secretary  of  the  Air  Force 
(Acquisition),  the  Air  Force  Program  Executive  Offices;  Air  Force  annual 
statements  of  assurance;  and  supporting  documentation.  We  reviewed  those 
documents  to  identify  whether  a  standardized  procedure  was  in  place  to  complete 

*  Acquisition  Category  III. 

^  Acquisition  Category  II. 
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and  submit  the  required  documentation  and  to  determine  whether  the  Air  Force 
had  identified  any  material  management  control  weaknesses  for  Air  Force 
Acquisition  Category  II  and  III  programs. 

To  evaluate  the  reporting  process  used  by  the  Air  Force  to  support  its 
management  control  of  acquisition  programs,  we  reviewed  documentation  dated 
from  June  1995  through  June  2004.  The  documentation  included  acquisition 
decision  memorandum,  APBs,  acquisition  strategy  reports,  affordability 
assessments,  analyses  of  alternatives,  C4I  support  plans,  PESHEs,  ORDs,  and 
TEMPs  for  the  eight  Air  Eorce  programs  selected  for  review.  We  reviewed  those 
documents  to  ensure  that  the  required  program  documentation  for  each  program 
was  complete,  accurate,  and  up-to-date. 

We  performed  this  audit  from  November  2003  through  June  2004  in  accordance 
with  generally  accepted  government  auditing  standards. 

Use  of  Computer-Processed  Data,  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

General  Accounting  Office  High-Risk  Area,  The  General  Accounting  Office 
has  identified  several  high-risk  areas  in  DoD.  This  report  provides  coverage  of 
the  DoD  Weapons  Systems  Acquisition  high-risk  area. 


Management  Control  Program  Review 


DoD  Directive  5010.38,  “Management  Control  (MC)  Program,”  August  26,  1996, 
and  DoD  Instruction  5010.40,  “Management  Control  (MC)  Program  Procedures,” 
August  28,  1996,  require  DoD  organizations  to  implement  a  comprehensive 
system  of  management  controls  that  provides  reasonable  assurance  that  programs 
are  operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  the  Review  of  the  Management  Control  Program,  In  accordance 
with  DoD  policy,  acquisition  managers  are  to  use  program  cost,  schedule,  and 
performance  parameters  as  control  objectives  in  APBs  to  implement  the 
requirements  of  DoD  Directive  5010.38.  Accordingly,  we  limited  our  review  to 
management  controls  directly  related  to  program  cost,  schedule,  and  performance 
in  APBs.  We  also  reviewed  the  adequacy  of  management’s  self-evaluation  of 
those  controls. 

Adequacy  of  Management  Controls,  We  identified  material  management 
control  weaknesses  for  the  Air  Eorce,  as  defined  in  DoD  Instruction  5010.40.  The 
Air  Eorce  Acquisition  Executive  and  the  Program  Executive  Officers  did  not 
ensure  that  controls  for  obtaining,  preparing,  updating,  and  approving  program 
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documentation  and  preparing  program  deviation  reports  were  suffieient  that 
program  managers  timely  provided  MDAs  with  aequisition  program 
documentation  needed  to  effectively  oversee  a  program’s  cost,  schedule,  and 
performance  data  at  and  between  milestone  decision  reviews. 

Reeommendations  1.  and  2.,  if  implemented,  will  improve  the  quality  and 
timeliness  of  program  information  provided  to  Air  Foree  MDAs  so  that  they  ean 
make  fully  informed  investment  decisions.  A  copy  of  the  report  will  be  provided 
to  the  senior  offieial  responsible  for  management  eontrols  in  the  Air  Foree. 

Adequacy  of  Management’s  Self-Evaluation,  None  of  the  Program  Exeeutive 
Offiees  or  program  offiees  reviewed  identified  management  eontrol  weaknesses 
related  to  preparing  and  updating  required  aequisition  program  doeumentation 
used  for  program  oversight  and  deeision  making.  The  Program  Exeeutive 
Offieers  and  program  managers  did  not  report  those  weaknesses  beeause  they  did 
not  reeognize  the  laek  of  up-to-date  doeumentation  as  a  management  eontrol 
weakness. 


Prior  Coverage 


During  the  last  5  years,  the  Air  Eoree  Audit  Ageney  issued  one  report  that 
addressed  management  eontrol  weaknesses  for  an  aequisition  program. 

Air  Force  Audit  Agency 

Report  No.  E2003-0002-ECQ000,  “Integrated  Broadeast  Serviee  Eleetronie 
Systems  Center,  Hanseom  AEB  MA,”  May  12,  2003 
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Appendix  B.  Management  Control  Objectives 

and  Techniques 


Control  Objectives.  Control  objectives  are  the  specific  goals,  conditions,  or 
levels  of  control  that  a  manager  establishes  for  an  assessable  unit  to  provide 
reasonable  assurance  that  the  resources  assigned  to  that  organization  are 
adequately  safeguarded  against  waste,  fraud,  and  mismanagement. 

Control  objectives  must  be  identified  for  each  organization  and  should  be  logical, 
applicable,  reasonably  complete,  and  tailored  to  an  agency’s  operations,  which 
means  that  the  control  objectives  should  be  the  positive  effects  that  management 
tries  to  attain  or  the  negative  effects  it  seeks  to  avoid  through  adherence  to 
established  management  controls. 

For  Air  Force  acquisition  programs,  basic  control  objectives  involve  the  program 
office’s  ability  to  adhere  to  a  weapon  system’s  cost,  schedule,  and  performance 
baseline  parameters  that  are  embodied  in  APBs.  The  program  manager  updates 
and  the  MDA  approves  the  APB  at  milestone  reviews.  As  weapon  systems 
progress  through  the  acquisition  process,  baseline  parameters  are  refined  and  the 
level  of  detail  evolves  at  succeeding  milestone  reviews.  The  program  manager 
identifies  minimum  acceptable  requirements  for  each  baseline  parameter,  known 
as  thresholds.  Values  for  APB  parameters  show  the  system’s  cost  and 
performance  characteristics  as  they  are  expected  to  be  produced  and  fielded,  as 
well  as  critical  acquisition  schedule  events.  If  those  minimum  acceptable  levels 
or  thresholds  are  not  met,  the  MDA  may  require  a  reevaluation  of  alternative 
concepts  or  design  approaches. 

Program  exit  criteria  are  also  control  objectives  for  Air  Force  acquisition 
programs.  Exit  criteria  are  the  specific  minimum  requirements  that  a  system  must 
satisfactorily  demonstrate  before  the  MDA  will  consider  approving  the  system  for 
transitioning  to  the  next  acquisition  phase. 

Control  Techniques.  Control  techniques  are  mechanisms  by  which  control 
objectives  are  achieved.  A  control  technique  is  any  form  of  organization, 
procedure,  or  document  flow  that  is  relied  on  to  accomplish  a  control  objective 
and  help  safeguard  or  protect  an  organization  from  fraud,  waste,  and 
mismanagement.  Control  techniques  include,  but  are  not  limited  to,  specific 
policies,  procedures,  plans  of  organization  (including  separation  of  duties),  and 
physical  arrangements  (such  as  locks  and  fire  alarms).  Management  control 
techniques  are  to  continually  provide  a  high  degree  of  assurance  that  the 
management  control  objectives  are  being  achieved. 
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For  Air  Force  acquisition  programs,  the  milestone  review  documentation 
specified  in  DoD  Instruction  5000.2,  “Operation  of  the  Defense  Acquisition 
System,”  May  12,  2003,  provides  control  techniques  to  achieve  the  control 
objectives.  The  management  control  techniques  in  the  Instruction  include  the 
documents  in  the  following  table. 

Management  Control  Techniques 


Control  Technique 

Obiective(s)  Controlled 

Prenared  Bv 

Submitted  To 

APB  Agreements 

Cost,  schedule,  and 
performance  baselines 

Program  Manager 

MDA 

Command,  Control, 
Computers, 
Communications, 
and  Intelligence 
(C4I)  Support  Plan 

C4I  support  requirements  to 
satisfy  the  development, 
testing,  and  operational 
employment  of  the  system 

DoD  Component 

MDA 

Earned  Value 
Management  Data 

Reports  the  progress  of  the 
program  in  meeting  cost 
parameters 

Contractor 

Program  Manager 

Operational 
Requirements 
Document  (ORD) 

Translates  mission  needs  into 
detailed  performance 
capabilities  and  characteristics 

DoD  Component 

Service 

Operational  Test 
Reports 

Reports  on  the  progress  of  the 
program  in  meeting 
performance  parameters 

Independent  Test 
Agency 

Program  Manager 

Test  and  Evaluation 
Master  Plan  (TEMP) 

Major  elements  and  objectives 
of  the  test  and  evaluation 
program 

Program  Manager 

Director, 
Operational  Test 
and  Evaluation;* 
MDA;  and  Air 
Force  Operational 
Test  and 

Evaluation  Center 

*The  Director,  Operational  Test,  and  Evaluation  approves  the  TEMP  for  all  test  and  evaluation  oversight 
programs  in  the  Office  of  the  Secretary  of  Defense. 


DoD  Instruction  5010.40,  “Management  Control  (MC)  Program  Procedures,” 
August  28,  1996,  requires  system  documentation  to  be  maintained  for 
management  control  programs.  System  documentation  includes  the  policies  and 
procedures,  organizational  charts,  manuals,  flow  charts,  and  related  written  and 
graphic  materials  necessary  to  describe  organizational  structure,  operating 
procedures,  and  administrative  practices  to  communicate  responsibility  and 
authority  for  accomplishing  programs  and  activities.  This  type  of  management 
control  documentation  is  specified  in  DoD  Instruction  5010.40. 
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Appendix  C.  Program  Documentation  Status 


DoD  Instruction  5000.2  identities  mandatory  program  doeuments  that  program 
managers  must  provide  at  program  milestone  reviews.  The  DoD  Interim  Defense 
Aequisition  Guidebook  explains  the  importanee  of  the  aequisition  program 
doeuments.  The  following  table  of  five  mandatory  aequisition  program 
doeuments  shows  whether  the  program  managers  for  the  eight  programs  reviewed 
provided  the  MDA  with  the  eompleted  and  approved  doeuments  at  the  aequisition 
program’s  last  milestone  deeision  review. 

Mandatory  Program  Documentation  Provided 


Program 

APB 

C4I 

Support 

Plan 

AFTERPS-R^ 

No 

No 

CCS-C^ 

No 

No 

CLOVerS^ 

Yes 

N/A 

FAB-T^ 

No 

No 

F-15E^ 

No 

No 

IBS^ 

Yes 

No 

mafd’ 

Yes 

No 

UMT^ 

No 

N/A 

ORD 

PESHE 

TEMP 

No 

No 

No 

Yes 

No 

No 

Yes 

No 

No 

Yes 

No 

No 

Yes 

No 

No 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 

N/A 

No 

*  Air  Force  Terminal  Instrument  Procedures-Replacement. 
^  Command  and  Control  System-Consolidated. 

^  Common  Low  Observable  Verification  Systems. 

Family  of  Advanced  Beyond-Line-of-Sight  Terminals. 

^  F-15E  Advanced  Display  Core  Processor  Program. 

^  Integrated  Broadcast  Service. 

^  Miniature  Air  Launched  Decoy. 

*  Universal  Munitions  Trailer. 
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Importance  of  Milestone  Documents 


Acquisition  Program  Baseline.  Every  acquisition  program  must  have  an  APB  at 
program  initiation.  The  program  manager  prepares  the  APB  in  coordination  with 
the  user  to  document  the  most  important  cost,  schedule,  and  performance 
parameters  (both  threshold  and  objective)  for  the  program.  The  program  manager 
bases  the  APB  on  user  performance  requirements,  schedule  requirements,  and  an 
estimate  of  total  program  cost.  The  program  manager,  in  coordination  with  the 
user,  should  obtain  approval  for  a  revised  APB  if  the  program  is  restructured  or  if 
there  is  an  unrecoverable  program  deviation. 

C4I  Support  Plan.  DoD  Components  must  develop  C4I  support  plans  for  all 
programs  early  in  the  acquisition  process  when  the  Components  connect  in  any 
way  to  the  communication  and  information  infrastructure.  Acquisition  decision 
makers  are  required  to  review  the  C4I  support  plan  at  each  program  milestone 
decision;  at  decision  reviews,  as  appropriate;  and  whenever  support  requirements 
change.  A  C4I  support  plan  assists  in  identifying  system  integration  issues  that 
need  to  be  addressed  during  testing. 

Operational  Requirements  Document.  The  ORD  is  a  formatted  statement 
containing  performance  parameters  for  the  proposed  concept  or  system  that  is 
prepared  by  the  user  or  the  user’s  representative.  An  ORD  provides  the 
developmental  and  operational  testers  with  the  ability  to  determine  whether  the 
system  meets  minimum  required  system  capabilities. 

Programmatic  Environmental  Safety  and  Health  Evaluation.  The  program 
manager  is  to  initiate  the  evaluation  at  the  earliest  possible  time  in  support  of  a 
program  initiation  decision  and  maintain  an  updated  evaluation  throughout  the 
life  cycle  of  the  program.  When  program  managers  perform  the  analyses  for  the 
PESHE,  they  gain  information  on  the  potential  environmental,  safety,  and  health 
effects  of  developing,  fielding,  storing,  demilitarizing,  and  disposing  of  their 
weapon  systems.  Information  should  be  obtained  before  the  program  enters  the 
system  development  and  demonstration  phase  to  prevent  program  delays  and 
added  program  costs  stemming  from  the  violation  of  environmental,  safety,  or 
health  laws. 

Single  Acquisition  Management  Plan.  The  single  acquisition  management  plan 
is  a  comprehensive,  integrated  plan  written  at  the  strategic  level,  which  discusses 
all  relevant  aspects  of  a  program. 

Test  and  Evaluation  Master  Plan.  The  TEMP  documents  the  overall  structure 
and  objectives  of  the  test  and  evaluation  program.  It  provides  a  framework  within 
which  to  generate  detailed  test  and  evaluation  plans  and  to  document  schedule 
and  resource  implications  associated  with  the  test  and  evaluation  program.  The 
TEMP  identifies  necessary  developmental  test  and  evaluation,  operational  test 
and  evaluation,  and  live-fire  test  and  evaluation  activities.  An  approved  TEMP  is 
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an  essential  tool  for  the  program  manager  and  operational  testers  to  identify 
resources  needed  to  perform  necessary  tests  and  to  test  all  key  performance 
parameters  and  resolve  problem  areas. 


Program  Documentation  Status 


Air  Force  Terminal  Instrument  Procedures-Replacement,  The  System 
Program  Director,  Global  Air  Traffic  Operations/Mobility  Command  and  Control 
approved  the  Air  Force  Terminal  Instrument  Procedures-Replacement  Program  to 
enter  the  production  and  deployment  phase  for  Release  A  and  the  system  and 
demonstration  phase  for  Release  B  on  April  2,  2003.  At  the  milestone  decision 
review,  the  program  office  did  not  have  an  approved  APB,  C4I  support  plan, 
ORD,  PESHE,  or  TEMP  for  MDA  review.  The  system  program  director 
approved  the  APB  on  May  12,  2004.  The  PESHE  was  approved  on  May  26, 

2004.  As  of  June  2004,  the  program  office  had  not  completed  or  obtained 
approval  of  the  C4I  support  plan,  ORD,  and  TEMP. 

The  program  office  developed  a  requirements  correlation  matrix  to  replace  the 
ORD  but  the  requirements  correlation  matrix  is  not  a  stand-alone  document.  The 
Joint  Integration  and  Test  Command  concluded  that  some  Air  Eorce  Terminal 
Instrument  Procedures-Replacement  system  interfaces  need  to  be  tested  and 
certified  so  that  the  program  can  meet  its  DoD  regulatory  obligations.  However, 
the  Joint  Integration  and  Test  Command  will  not  certify  the  Air  Eorce  Terminal 
Instrument  Procedures-Replacement  system  without  a  Joint  Staff-certified  ORD. 
Because  the  program  does  not  have  a  certified  ORD,  the  program  office 
unilaterally  determined  that  Joint  Integration  and  Test  Command  certification  was 
not  required.  Accordingly,  the  system  program  director  requested  that  the  Air 
Eorce  Communications  Agency  waive  the  C4I  support  plan  requirement  because 
the  requirements  correlation  matrix  does  not  include  an  interoperability  key 
performance  parameter.  Because  the  program  office  did  not  believe  that  a  TEMP 
was  required  for  an  Acquisition  Category  III  program,  it  developed  a  test  and 
evaluation  concept  plan  dated  March  19,  2003. 

Command  and  Control  System-Consolidated.  The  Command  and  Control 
System-Consolidated  Program  Office  did  not  have  an  approved  acquisition 
decision  memorandum  to  document  the  approval  to  enter  the  system 
development  and  demonstration  phase  of  the  acquisition  process.  The  Command 
and  Control  System-Consolidated  entered  the  system  development  and 
demonstration  phase  in  Eebruary  2001.  Eurther,  the  program  office  did  not  have 
an  approved  APB,  C4I  support  plan,  PESHE,  or  TEMP.  As  of  June  2004,  the 
program  office  had  not  completed  or  obtained  approval  of  the  APB,  C4I  support 
plan,  PESHE,  and  TEMP.  The  ORD  identified  system  interoperability 
requirements  in  the  C4I  support  plan  as  a  key  performance  parameter.  The 
program  office  prepared  the  C4I  support  plan  and  the  PESHE  and  should  be 
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approved  by  fall  2004.  The  TEMP  was  projeeted  to  be  eompleted  by  June  2004, 
and  the  APB  is  in  eoordination. 

Common  Low  Observable  Verification  System,  The  Commander, 
Aeronautieal  Systems  Center  approved  the  Common  Low  Observable 
Verifieation  System  to  enter  the  engineering,  manufaeturing,  and  development 
phase  of  the  aequisition  proeess  on  May  6,  1999.  At  the  milestone  deeision 
review,  the  program  offiee  did  not  have  an  approved  PESHE  or  TEMP  for  MDA 
review.  The  PESHE  was  not  approved  until  January  2003,  3.5  years  after  the 
milestone  decision  review.  Instead  of  preparing  a  TEMP,  the  program  office 
developed  a  broad,  top-level  test  strategy  in  the  single  acquisition  management 
plan. 

F-15E  Advanced  Display  Core  Processor,  The  P-15E  Advanced  Display  Core 
Processor  Program  Office  did  not  have  an  approved  acquisition  decision 
memorandum  to  document  the  approval  to  enter  the  system  development  and 
demonstration  phase  of  the  acquisition  process.  The  P-15E  Advanced  Display 
Core  Processor  entered  the  system  development  and  demonstration  phase  on 
June  8,  1999.  At  that  time,  the  program  office  did  not  have  an  approved  APB, 
C4I  support  plan,  PESHE,  or  TEMP.  As  of  June  2004,  the  documents  had  not 
been  completed  or  approval  obtained  because,  according  to  the  program  office, 
the  TEMP  and  the  C4I  support  plan^  were  not  required  for  an  Acquisition 
Category  III  program.  Additionally,  the  program  office  stated  that  it  did  not  have 
to  develop  a  C4I  support  plan  because  the  Advanced  Display  Core  Processor  did 
not  have  any  external  interfaces.  Eurther,  the  program  office  considered  the 
single  acquisition  management  plan  as  a  substitute  for  the  APB  and  had  no  plans 
to  develop  and  obtain  approval  of  an  APB. 

The  program  office’s  statements  that  a  C4I  support  plan  was  not  required  because 
the  P-15E  Advanced  Display  Core  Processor  was  an  Acquisition  Category  III 
program  and  it  did  not  have  any  external  interfaces  is  contrary  to  policy  in  both 
CJCS  Instruction  6212. OIB,  “Interoperability  and  Supportability  of  National 
Security  Systems,  and  Information  Technology  Systems,”  May  8,  2000,  and  its 
revision,  CJCS  Instruction  6212. OIC,  “Interoperability  and  Supportability  of 
Information  Technology  and  National  Security  Systems,”  November  8,  2003. 
Specifically,  CJCS  Instruction  6212. OIB  states  that  the  C4I  support  plan  is 
required  for  all  systems,  regardless  of  acquisition  category,  to  identify 
dependencies  and  interface  requirements  between  systems.  Similarly,  CJCS 
Instruction  62 12.0 1C  requires  the  Information  Support  Plan,^  regardless  of 
acquisition  category,  for  all  developmental  and  fielded  systems  to  permit 
evaluation  of  system  interoperability  and  supportability  requirements. 
Additionally,  although  the  Advanced  Display  Core  Processor  does  not  directly 
interface  with  external  systems,  it  does  enhance  the  overall  interoperability  of  the 
P-15E  aircraft.  As  stated  in  the  ORD,  the  Advanced  Display  Core  Processor  will 


*  C4I  support  plan  now  designated  as  the  Information  Support  Plan  in  CJCS  Instruetion  6212. OIC. 
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allow  the  F-15E  to  receive  and  display  various  types  of  information,  including 
correlated  airborne  and  ground  tracks,  and  with  other  fighter  aircraft  and  sensor 
platforms  (such  as  the  Airborne  Warning  and  Control  System). 

CJCS  Instruction  6212. OIB  specifically  states  that  hardware  and  software 
modifications  that  affect  the  interoperability  of  fielded  systems  will  require  Joint 
Interoperability  Test  Command  recertification  before  the  system  modifications 
are  fielded  for  initial  operational  capability.  The  Joint  Interoperability  Test 
Command  uses  the  C4I  support  plans  to  support  the  recertification  of  modified 
systems.  Similarly,  CJCS  Instruction  62 12.0 1C  requires  the  Information  Support 
Plan  to  support  interoperability  certification  for  modified  systems.  As  of  June 
2004,  the  Joint  Interoperability  Test  Command  had  not  certified  that  the  F-15 
platform  met  DoD  interoperability  requirements  because  the  F-15  program  office 
had  not  prepared  a  C4I  support  plan  for  the  overall  F-15  system,  which  would 
include  the  Advanced  Display  Core  Processor  and  the  F-15  platform. 

Instead  of  preparing  a  TFMP,  the  program  office  addressed  the  test  strategy  in  the 
single  acquisition  management  plan.  The  F-15  System  Program  Office  prepared 
the  PFSHE  in  January  2003  but,  as  of  June  2004,  it  still  had  not  been  approved. 

Family  of  Advanced  Beyond-Line-Of-Sight  Terminals.  The  Family  of 
Advanced  Beyond-Fine-Of-Sight  Terminals  Program  Office  did  not  have  an 
approved  acquisition  decision  memorandum  to  document  the  approval  to  enter  the 
system  development  and  demonstration  phase  of  the  acquisition  process.  The 
Family  of  Advanced  Beyond-Fine-Of-Sight  Terminals  entered  the  system 
development  and  demonstration  phase  on  September  20,  2002.  At  that  time,  the 
program  office  did  not  have  an  approved  APB,  C4I  support  plan,  PFSHE,  or 
TEMP.  As  of  June  2004,  the  documents  had  not  been  completed  or  approval 
obtained.  A  new  program  manager  took  over  in  November  2003,  at  which  time  a 
review  of  all  major  program  documents  was  conducted  and  the  decision  was  made 
to  update  the  documents  when  the  program  becomes  an  Acquisition  Category  I 
program  sometime  during  the  summer  of  2004.  According  to  the  deputy  program 
manager,  the  status  of  documents  was  briefed  to  the  MDA  and  the  Program 
Executive  Officer  at  the  November  2003  program  management  review. 

Integrated  Broadcast  Service,  The  Principal  Deputy  Assistant  Secretary  of  the 
Air  Force  (Acquisition  and  Management)  approved  the  Integrated  Broadcast 
Service  to  enter  the  engineering,  manufacturing,  and  development  phase  of  the 
acquisition  process  on  May  2,  2001.  At  the  milestone  decision  review,  the 
program  office  did  not  have  an  approved  C4I  support  plan  for  MDA  review.  As 
of  June  2004,  the  program  office  had  prepared  a  C4I  support  plan  but  was  waiting 
for  review  and  further  guidance  from  the  Air  Force  Communications  Agency. 

Miniature  Air  Launched  Decoy,  The  Commander,  Air  Armament  Center 
approved  the  Miniature  Air  Faunched  Decoy  Program  to  enter  the  system 
development  and  demonstration  phase  on  January  6,  2003.  At  the  milestone 
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decision  review,  the  program  office  did  not  have  an  approved  TEMP,  C4I  support 
plan,  and  PESHE  for  MDA  review.  The  TEMP  was  not  approved  until  March  3, 

2003,  2  months  after  the  milestone  decision  review.  The  program  office  provided 
a  C4I  support  plan  dated  April  2003  that  had  not  been  approved,  but  planned  to 
have  a  completed  and  approved  PESHE  by  May  2006. 

Universal  Munitions  Trailer,  The  Universal  Munitions  Trailer  Program  Office 
did  not  have  an  approved  acquisition  decision  memorandum  to  document  the 
approval  to  enter  the  system  development  and  demonstration  phase  of  the 
acquisition  process.  The  Universal  Munitions  Trailer  program  entered  the  system 
development  and  demonstration  phase  on  September  10,  2002.  At  that  time,  the 
program  office  did  not  have  an  approved  APB,  ORD,  and  TEMP.  As  of  June 

2004,  the  program  office  had  not  completed  and  obtained  approval  of  a  TEMP  or 
obtained  an  approved  ORD.  The  program  office  developed  a  requirements 
correlation  matrix  to  replace  the  ORD,  but  the  requirements  correlation  matrix  is 
not  a  stand-alone  document.  Instead  of  preparing  a  TEMP,  the  program  office 
developed  a  draft  combined  developmental  and  operational  test  and  evaluation 
plan  because  the  program  office  believed  that  a  TEMP  was  not  required  for  an 
Acquisition  Category  III  program. 
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Appendix  D.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Under  Secretary  of  Defense  (Comptroller)/Chief  Financial  Officer 
Director,  Program  Analysis  and  Evaluation 
Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Director,  Operational  Test  and  Evaluation 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Assistant  Secretary  of  the  Air  Force  (Acquisition) 

Program  Executive  Officer,  Aircraft  Systems 

Program  Manager,  Common  Fow  Observable  Verification  System 
Program  Manager,  F-15E  Advanced  Display  Core  Processor 
Program  Executive  Officer,  Command  and  Control  &  Combat  Support 

Program  Manager,  Air  Force  Terminal  Instrument  Procedures-Replacement 
Program  Manager,  Integrated  Broadcast  Service 
Program  Executive  Officer,  Space 

Program  Manager,  Command  and  Control  System-Consolidated 
Program  Manager,  Family  of  Advanced  Beyond-Fine-of-Sight  Terminals 
Program  Executive  Officer,  Weapons 

Program  Manager,  Miniature  Air  Faunched  Decoy 
Program  Manager,  Universal  Munitions  Trailer 
Auditor  General,  Department  of  the  Air  Force 
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Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 


Senate  Committee  on  Appropriations 

Senate  Subeommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Serviees 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subeommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Serviees 
House  Committee  on  Government  Reform 

House  Subeommittee  on  Government  Effieieney  and  Finaneial  Management,  Committee 
on  Government  Reform 

House  Subeommittee  on  National  Seeurity,  Emerging  Threats,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subeommittee  on  Teehnology,  Information  Poliey,  Intergovernmental  Relations, 
and  the  Census,  Committee  on  Government  Reform 
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Team  Members 

The  Office  of  the  Deputy  Inspector  General  for  Auditing  of  the  Department  of 
Defense,  Acquisition  Management  prepared  this  report.  Personnel  of  the  Office 
of  the  Inspector  General  of  the  Department  of  Defense  who  contributed  to  the 
report  are  listed  below. 

John  E.  Meling 
Celia  J.  Harrigan 
Kevin  B.  Palmer 
Lidet  K.Negash 
Deanna  Wyatt 
Jacqueline  N.  Pugh 


